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Abstract. Teacher education in Indonesia aims to prepare pre-service teachers in becoming 
professional teaching personnel. Two important knowledge need to be acquired are content 
knowledge (CK) and pedagogical content knowledge (PCK). However, it is still a challenge to 
measure the pre-service mathematics teachers’ CK and PCK. Therefore, the first aim of the 
present study is to develop and validate a test instrument to measure CK and PCK of pre-service 
mathematics teachers on high school level mathematics. Second, the present study aims to 
measure the CK and PCK of pre-service mathematics teachers by using the developed 
instrument. The present study employed developmental research method and used students’ 
empirical data in investigating the validity, objectivity, and reliability of the instrument. The 
results of the study and its implications for future research and practice then will be presented in 
the paper. 
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1. Introduction 

Teacher education in Indonesia aims to prepare pre-service teachers in becoming professional teaching 
personnel. However, concerns have been arisen with regard to the effectiveness of the program in 
equipping them with knowledge for teaching to provide high-quality teaching and learning for their 
prospective students. Two important knowledges needed by the pre-service teachers is content 
knowledge (CK) and pedagogical content knowledge (PCK) [1]. If those knowledges are possessed by 
the teachers, then they potentially can deliver high-quality instruction. As a result, their students will 
acquire high academic achievements [2, 3, 4]. 

CK and PCK need to be acquired by pre-service teachers in general, but also by pre-service 
mathematics teacher in particular. Within mathematics education, the concepts of CK and PCK are 
commonly known as mathematical knowledge for teaching (MKT) or content knowledge for teaching 
mathematics (CKTM) [5]. The MKT is critical component for pre-service mathematics teacher in their 
preparation and professional development. With this knowledge, for example, they can arrange the 
sequence through which their students learn mathematics effectively. In addition, with this knowledge, 
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they can present mathematics instruction in which suitable for the development and prior knowledges 
of their students. For these reasons, MKT become one of critical aspects for pre-service mathematics 
teachers in preparing mathematics teaching and learning. As an evident, Hatisaru and Erbas [6] found 
that MKT influence the teaching practices. As consequences, the teaching practices have positive role 
in student learning. The positive impact of MKT on teaching practices and student learning also has 
been found in recent studies [e.g., 7, 8, 9, 10]. 

Given the importance of CK, PCK, or MKT for pre-service mathematics teachers, it is necessary to 
measure their CK, PCK, or MKT. However, it still be a challenge to measure pre-service teachers’ 
knowledge in teaching mathematics even though many studies have attempted to develop CK and PCK 
scale in mathematics [e.g., 11, 12]. Therefore, the aims of the present study are twofold. First, the present 
study aims to develop a reliable, objective, and valid instrument in measuring pre-service mathematics 
teachers’ CK and PCK. Second, the present study also measures CK and PCK of the pre-service 
mathematics teachers using the developed instrument. 


2. Literature review 


2.1. CK, PCK, and MKT 

The notion of CK and PCK were proposed by Shulman [1]. CK and PCK are two categories in 
Shulman’s typologies for characterizing professional knowledge for teaching. However, the typologies 
are articulated variously across the literature. One of complete articulations is presented by Ball et al. 
[5]. Based on this articulation, there are seven main categories of teacher knowledge, namely (1) general 
pedagogical knowledge, (2) knowledge of students and their characteristics, (3) knowledge of 
educational context, (4) knowledge of educational ends, purposes, and values, and their philosophical 
and historical grounds, (5) content knowledge, (6) curriculum knowledge, (7) and pedagogical content 
knowledge. Two teacher knowledge categories are emphasized in this article, i-e., content knowledge 
and pedagogical content knowledge. Content knowledge referred to knowledge about the subject and 
its organizing structures. For pedagogical content knowledge, Shulman [1] identified two central 
components, namely knowledge about instructional methods and representations and knowledge 
regarding students’ (mis)conceptions. 
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Figure 1. The relationship of CK, PCK, and MTK [13] 


Aside of Shulman’s central contribution on conceptualizing teacher knowledge, it also has been 
criticized [13]. Therefore, many scholars develop alternative conceptualizations. In mathematics 
education, Ball et al. [5], Hill et al. [11, 14], and Hill, et al. [7] reconceptualized teachers’ PCK as 
mathematical knowledge for teaching (MKT). This reconceptualization covers both CK and PCK. MKT 
covers six domains of mathematical knowledge of teaching, i.e. common content knowledge, horizon 
content knowledge, specialized content knowledge, knowledge of content and students, knowledge of 
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content and teaching, and knowledge of content and curriculum. The first three categories of MKT are 
related to teachers’ CK whereas the remaining categories deal with teachers’ PCK. The relationship of 
Shulman’s [1] conceptualization of teacher knowledge and Ball et al.’s [5] MKT is shown in Figure 1. 


2.2. Measuring CK and PCK 
Section 2.1. shows that Ball et al.’s [5] MKT relates to Shulman’s [1] conceptualization of CK and PCK. 
Therefore, in the rest of this article we will use the term CK and PCK. 

There are various subscales which can be used to measure CK and PCK in literature. In present study, 
we use the CK and PCK subscales from the work of Krauss et al. [15]. The subscales are in line with 
our theoretical framework of PCK presented in Section 2.1. First PCK subscale is knowledge of 
mathematical tasks. The mathematical-task knowledge is included in PCK subscales because 
mathematical tasks are central to mathematics teaching and learning [16, 17]. Furthermore, there is 
evidence in literature [e.g. 18] showing that there is positive association between this knowledge and 
teachers’ PCK. Second PCK subscale is knowledge of student misconceptions and difficulties. It is 
important for teachers to be aware of students’ misconception because it can be used to acquire 
knowledge of students’ thinking [19]. Knowing students’ difficulties in mathematics is also critical 
because it can be used for teachers to help students so that they be able to pursue mathematics further 
[20]. Furthermore, the knowledge construction by students often succeeds with the guidance and support 
by teacher [21, 22]. Thus, the knowledge of mathematics-specific instructional strategies is third 
component of PCK subscales. 

The subscale in measuring CK is background knowledge on the contents of the high school level 
mathematics curriculum. Pre-service mathematics teachers should master mathematics contents beyond 
the students should do. It means that the pre-service mathematics teachers must possess mathematical 
knowledge that much deeper than their prospective students. 


3. Method 

The developmental research method was employed in developing test items. The three parts of the study 
reported here are (1) conceptualization of test scales and subscales, (2) development of the CK and PCK 
items, and (3) validation and quantitative analysis of the developed CK and PCK items. From the step 
1, we obtained three subscales in measuring pre-service mathematics teachers’ PCK, namely knowledge 
of mathematical tasks (Task), knowledge of student misconceptions and difficulties (Student), and 
knowledge of mathematics-specific instructional strategies (Instruction). For CK, we conceived as a 
background knowledge on the contents of the high school level mathematics curriculum. 

In step 2, we developed CK and PCK items based on the CK and PCK subscales, along with their 
scoring rubrics. We composed two items for each subscale. The mapping between subscales and items 
is shown in Table 1. In total, there were ten items. The items 1.1, 2.1, 3.1, 4.1, and 5.1 were put in first 
test instrument, whereas the remaining items were put in second test instrument. Example of one item 
in the items set is shown in Figure 2. 


Table 1. Mapping between CK and PCK subscales and items 


Subscale and indicator Item 
PCK Task 
Giving or evaluating mathematical explanations 1.1 


Appraising and adapting the mathematical content of textbooks or other resources 1.2 

PCK Student 
Giving examples and illustrations of possible student’s misconceptions and 2.1 and 2.2 
difficulties in solving mathematical problem. 

PCK Instruction 
Responding to students’ “why” questions by using language that suitable with 3.1 and 3.2 
student’s development 

CK 
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Solving non-routine mathematical problem 4.1, 5.1, 4.2, 
and 5.2 


The development of CK and PCK items were followed by the validation process. In this phase, the 
CK and PCK test instrument were distributed to two experts along with the validation sheet. The 
validation sheet contained four criteria which be used in assessing the CK and PCK items, i.e. goal- 
centered criteria, learner-centered criteria, context-centered criteria for assessments, and assessment- 
centered criteria [23]. Aside of these criteria, the expert validators were allowed to give comments on 
the validation sheet or directly on the test instrument. 


Soal Nomor 3: Rerata Data Tunggal dan Berkelompok 


Ada seorang siswa bertanya seperti ini: Bu, PR kemarin memberikan data seperti 
ini. 

TO 4 48 

53 6 40 

52 - 56 

33 46 
Saya menghitung rerata data tersebut dengan rumus jumlah nilai dibagi dengan 
banyak nilai dan saya memperoleh hasil seperti berikut. 


Tetapi, teman saya menghitung rerata data tersebut dengan terlebih dahulu 
mengubahnya menjadi data berkelompok seperti ini. 


Kelas Frekuensi (fi) Nilai Tengah (xi) Si * xi 
31-40 5 3555 
41-50 5 45,5 
51-60 55;5 
61-70 65,5 
20 
Dengan demikian, rerata data tersebut adalah 


Mengapa jawaban saya dan teman saya berbeda? Jawaban mana yang betul, Bu? 


Bagaimana Anda menanggapi pertanyaan-pertanyaan siswa tersebut? 


Figure 2. Example of an item from PCK Instruction subscale (in Indonesian) 


The validation phase was then followed by the revision of items. The revision process was based on 
the validator’s evaluation. After revision, the first test instrument then distributed to the participants to 
be answered and then the answer be assessed by three independent correctors. The three correctors were 
master students who were also serve as research assistant. 
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3.1, Participants 

The participants of the present study were fifteen pre-service mathematics teachers in teacher education 
program of Universitas Sanata Dharma, Yogyakarta. Of the fifteen pre-service teachers, 14 (93.3%) are 
female and 1 (6.7%) is male. Their participation in the present study was one of their evaluation in the 
aforementioned program. 


3.2. Data Analysis 
The present study used participants’ answers to analysis psychometric properties of CK and PCK test 
instrument as well as to evaluate psychometric items statistics. Psychometric properties of CK and PCK 
test instrument included internal consistency and distribution of test scores, whereas the psychometric 
items statistics contained item discrimination indices, item difficulties, and item variances. Furthermore, 
objectivity of the developed rubrics was evaluated by Kendall’s coefficient of concordance [24]. 

The experts’ validation data also were used in evaluating the validity of the test instrument. The result 
of the validation contains quantitative and qualitative data. The quantitative data were analysed by using 
percentage criteria, whereas the qualitative data were used to make revision on the items set. 


4. Results 


4.1. Psychometric Properties of CK and PCK Items Set 

The reliability of the developed PCK and CK test instrument was determined by using Cronbach’s alpha 
(a) values. The items set showed acceptable reliability, with Cronbach’s alpha of @ = .53. The item-total 
correlations showed that the items determined adequately. The mean of the correlations is M = .59, and 
the correlations are ranging from .28 to .72. The intercorrelations of the items is given in Table 2. Based 
on the intercorrelations value, the PCK Task item is less reliable than the other items. The correlations 
between PCK Task and other items are quite small, ranging from -.10 to .18. 


Table 2. Intercorrelations of pre-service mathematics teachers’ CK and PCK 


Correlations Task Student Instruction cky ckp 
PCK 

Task 

Student 18 

Instruction -.10 10 
CK 

ck; -.03 15 36 

cko 01 58 12 54 
M 2.47 3.16 1.60 1.16 91 
SD .60 73 1.00 72 65 


A Kolmogorov-Smirnov test was performed in evaluating the distribution of CK and PCK total 
scores or the distribution of item difficulties. The Kolmogorov-Smirnov test confirmed that the total 
scores are from a normal distribution population (p > .15). This result shows that the CK and PCK test 
can distinguish between high, middle, and low achievers. Moreover, all items showed valid item 
difficulties (Pm) within the acceptable range, .20 < Pm < .80. 

The objectivity of item scoring was evaluated by using Kendall’s coefficient of concordance on the 
inter-rater agreement. The values of Kendall’s coefficient of concordance are ranging from 17.87 to 33 
and the p-values are ranging from .003 to .21. This result indicates that all raters are agree in their 
assessment, except on PCK Task item. 


4.2. Results from Expert Validation 
An expert validation was performed to evaluate the validity of the CK and PCK items. The quantitative 
data from this validation confirmed a satisfactory validity of the items. The mean scores of each criteria 
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of validity are 4 to 4.08 out of 5. Aside of quantitative data, this validation obtained comments from the 
validator. In summary, the comments were regard to the clarity of sentences structure used in each item 
and scoring rubrics, as well as the suggestions on alternative solution of each item. All these comments 
were used in revising the items draft. 


4.3. Pre-Service Mathematics Teachers’ CK and PCK 

In evaluating pre-service mathematics teachers’ CK and PCK, mean and standard deviation of their 
score are determined. The mean and standard deviation of the score are Mcx = 1.03, SDcx = .60, Mpcx 
= 2.41, and SDpcx = .48. The maximum possible score is 4. The results indicate that, in general, the pre- 
service mathematics teachers’ CK score are lower than the PCK score. 


: R? Linear = 0.156 


Pedagogical Content Knowledge 


1.50 


.00 50 1.00 1.50 2.00 


Content Knowledge 


Figure 3. Scatter plot of participants’ CK and PCK 


Figure 3 shows the scatter plot of participants’ CK and PCK. The CK score is the average of ck; and 
ck» items, whereas the PCK score is the average of PCK Task, PCK Student, and PCK Instruction. From 
the correlation analysis, it is obtained that r = .40 and p = .15. Even though the correlation value is 
moderate, but the p-value shows that the correlation is not statistically significant. 


5. Discussion 

The present study is a respond for the need of development of test instrument to measure pre-service 
mathematics teachers’ CK and PCK [e.g., 25]. Three major steps have been conducted in developing 
such instrument, from deriving CK and PCK subscales from literature, constructing test items, to 
validating the items set. The first aim of the present study is to develop valid, objective, and reliable CK 
and PCK test instrument. The validity of the test instrument was evaluated by using the item-total 
correlations, Kolmogorov-Smirnov test of the scores’ distribution, and item difficulties. Based on these 
analyses, it is found that, in general, the validity of the test instrument is acceptable. The objectivity of 
the scoring rubrics is found to be adequate. It also found that the test instrument is reliable. 

From the result of pre-service mathematics teachers’ CK and PCK measurement, it is shown that the 
PCK score is higher than the CK score. It indicates that the pre-service teachers who participated in 
present study have more knowledge in mathematical task for teaching, students’ misconceptions and 
difficulties, and mathematics instruction than the knowledge about mathematics. This finding should be 
concern for mathematics educators on preparing future mathematics teachers. Shulman [1] emphasized 
that the teachers should possess content knowledge to in conducting teaching and learning. 

Due to the small size of our sample, it not shown that there is statistically significant association 
between pre-service mathematics teachers’ CK and PCK. This result not in line with many studies in 
literature [e.g. 26, 27]. Those studies found the connection between mathematical knowledge and 
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pedagogical content knowledge. Therefore, the pre-service mathematics teachers should be educated 
both from content knowledge and pedagogical content knowledge aspects, as stated by Shulman [1], 
“mere content knowledge is likely to be as useless pedagogically as content-free skill.” 


6. Conclusion 

The aims of the present study are to develop valid, objective, and reliable test instrument to measure 
pre-service mathematics teachers’ CK and PCK, and to measure pre-service mathematics teachers’ CK 
and PCK with the developed instrument. Therefore, the present study has been produced the test 
instrument that can used to quantify pre-service mathematics teachers CK and PCK. Aside of that, the 
present study also suggests that mathematics educators should educate the mathematics teacher 
candidate so that they possess mathematical knowledge and pedagogical content knowledge, even 
though the need is more on mathematical knowledge. 
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